Vision-Assisted Robotized Depalletizer

ABSTRACT

A vision-assisted robotized depalletizer receives at a pallet station a pallet loaded with products and outputs on a conveyor unitized products aligned in a desired orientation. Such depalletizers include a vision system having one or more image acquisition sensors that are positioned relative to the pallet station so as to have a field of view covering a top portion of the pallet. The sensors send to a controller one or more image of the top portion of the pallet that the controller uses along to determine the position of each product in the top portion of the pallet. The depalletizer further includes a robot equipped with a depalletizing tool that is coupled to the vision system controller for receiving information indicative of the position of each product from the top portion and uses that information to pick and position on the output conveyor each product from the top portion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.61/930,107, filed on Jan. 22, 2014, the contents of which areincorporated by reference herein.

FIELD

The present disclosure relates to depalletizing, and more specificallyto vision-assisted robotized depalletizing of products.

BACKGROUND

Distribution centers and warehouses typically receive their productssuch as cases, boxes, open trays, stretch wrapped trays, etc. on astructured pallet, e.g. orderly positioned without gaps between them.Depalletizing systems are known in the art to remove the products fromthe pallet.

Technologies known as “bulk depalletizers” are used to remove a completelayer of products by pushing them from the pallet to an outfeedconveyor. In this situation, each layer of product is typicallyseparated with a slip sheet.

Sometimes the products need to be depalletized and singulated so theycan be stored in an automatic storage and retrieval system. There areusually two approaches to do this. The first is manual depalletizing,where operators pick the products individually and forward them on anoutfeed conveyor. The other approach is to use a robot arm equipped witha layer depalletizing tool to remove the products layer by layer anddrop them on an outfeed conveyor. Products are singulated after beingdepalletized in order to being stored in an automatic storage andretrieval system.

Robotized layer depalletizing tools are quite efficient when a largeamount of products are found on each layer (small to medium sizedproducts). Because of the complexity of the tool and the weight of theload, the cycle time is long and the efficiency (products per minute)drops when the products are large. In this later case, picking productsone by one becomes a better approach. Also, with this approach, theproducts do not have to be singulated downstream as products are pickedone by one from the pallet. This yields that the complete footprint isreduced compared to the approach that includes the layer depalletizercoupled with a singulator.

U.S. Pat. No. 8,248,620 B2, issued on Aug. 21, 2012 to Wicks and Maueand being titled “Object Detection Device” describes an object detectingdevice for detecting the existence and general placement of an objectresiding upon a surface. It uses a laser measuring scanner disposed on alinear actuator to allow the depalletizer to properly center adepalletizer tool in relation to the top layer to be picked from theload.

The device from Wicks and Maue uses a vision system to detect the toplayer position only (e.g. outside corners of the top layer) and isunable to locate and position individual products on the layer.

U.S. Pat. No. 8,315,739 B2, issued on Nov. 20, 2012 to Dai and beingtitled “Determining the Position of an Object” describes a method fordetermining the position of at least one object present within a workingrange of a robot wherein an image is generated during a motion of thecamera mounted on a robot.

The method from Dai is directed to known objects that are randomlystacked in a container, and does not allow to identify boxes that areadjacent to each other. Indeed, Dai's method looks for pieces with knowndimensions that are distanced from each other and would not be operablewhen looking for products with no gaps therebetween. Also, in Dai, thecamera is mounted on a robot. This slows down the process consideringthat the robot has to stop moving to take an image and wait for theresult of the processing.

U.S. Pat. No. 6,332,750 B1, issued on Dec. 25, 2001 to Donner at al. andbeing titled “Method and Apparatus for Depalletizing Commodities”describes an apparatus for removing commodities from an upper side of asupport, including means for monitoring the position of the support andat least one of the implement (e.g. slip sheet) and the selectedcommodity. A position sensor is located in the moving mean (e.g. tool).

The apparatus from Donner works by implying that slip sheets arepresent, which is not always the case. Also, similarly to the system byDai, the position sensor is located on the tool, which impliesimmobilizing the tool frequently.

United States Patent Application No. 2014/0205403 A1, issued on Jul. 24,2014 to Criswell and being titled “Automated Truck Unloader forUnloading/Unpacking Product from Trailers and Containers” describes anapparatus and method for unloading product from a trailer by executing aroutine where a protruding product is identified and then unloaded by anindustrial robot.

The approach taken by Criswell is based on the presence of a protrudingproduct that can be identified by the vision system and therefore isinapplicable for depalletizing pallets where there are no protrudingproducts when a complete layer is presented at the cell.

European Patent No. 0 911 603 B1, issued on Oct. 22, 1997 to Ersü andbeing titled “Method for the Determination of the Position of aThree-Dimensional Body” describes a method for the optical determinationof the position of a finite rigid body where a selection of at leastthree lines of edges provided on the rigid body are identified.

The method from Ersü is applicable when the product to be positioned isalready singulated from other products and as such cannot be used forproducts that need to be depalletized.

The PCT Patent Application Publication No. WO 2006117814 A1, publishedon Nov. 9, 2006 to Lodi Rizzini and being titled “AutomaticDepalletizer” describes an apparatus and a method for depalletizing apallet of blocks of material, where an optic sensor detects thedistribution of the blocks located on the pallet before the blocks arepicked and conveyed to the unloading station.

The depalletizer by Lodi Rizzini is aimed at depalletizing blocks ofmaterials and is not suitable to depalletizing a wide variety ofproducts such as boxes, cases, trays, etc. The optic sensor described byRizzini is not versatile enough to detect and localize a wider varietyof products, for example of any color and of variable rectangular shape.

A robotic system that allows depalletizing products as various as cases,boxes, trays and the like that are orderly positioned on a pallet withor without gaps between the products is thus desirable.

SUMMARY

According to an illustrated embodiment, there is provided a

depalletizer comprising:

a pallet unloading station for receiving a pallet of products;

a vision system for taking at least one image of a top portion of thepallet received at the pallet station, and for using said at least oneimage for determining the position of said each product in the topportion of the pallet; and

a robot equipped with a depalletizing tool; the robot being coupled tothe vision system for receiving information therefrom indicative of theposition of said each product from the top portion and for using thatinformation to pick and position on an output surface said each productfrom the top portion.

According to another illustrative embodiment, there is provided a

depalletizer comprising:

a vision system including at least one image acquisition sensor fortaking an image of a top portion of a pallet positioned under the atleast one image acquisition system, and a vision system controllercoupled to the at least one image acquisition sensor for receiving theimage and for using said image to determine the position of each productin the top portion of the pallet; and

a robot equipped with a depalletizing tool including at least onesuction means; the robot being coupled to the vision system controllerfor receiving information therefrom indicative of the position of eachproduct from the top portion and for using that information to pick andposition on an output conveyor said each product from the top portion.

According to still another embodiment, there is provided a method fordepalletizing a pallet of products, the method comprising:

taking an image of a top portion of the pallet;

the controller using the image to determine a position of a product topick from the top portion of the pallet;

the controller sending to a robot information indicative of the positionof the product to pick on the pallet; and

the robot picking the product and moving the product on an outputsurface.

Considering the growing needs of warehouses and distribution centers forautomation, it is believed that there is a need for a vision-assistedrobotized depalletizer as described herein.

The term “image” should be construed in the description and in theclaims as including without limitations a color or a gray intensity ordepth map.

Other objects, advantages and features of vision-assisted robotizeddepalletizer will become more apparent upon reading the following nonrestrictive description of illustrated embodiments thereof, given by wayof example only with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the appended drawings:

FIG. 1 is a perspective view of a vision-assisted robotized depalletizeraccording to a first illustrative embodiment; the depalletizer beingshown with a full pallet therein;

FIG. 2 is a perspective view of the depalletizer from FIG. 1, shownafter a first product has been removed from the pallet;

FIG. 3 is a perspective view of the depalletizer from FIG. 1, thedepalletizer being shown after the first three products were removedfrom the pallet; and

FIG. 4 is a flowchart illustrating an embodiment of a vision-assistedmethod for depalletizing products on a pallet.

DETAILED DESCRIPTION

In the following description, similar features in the drawings have beengiven similar reference numerals, and in order not to weigh down thefigures, some elements are not referred to in some figures if they werealready identified in a precedent figure.

The use of the word “a” or “an” when used in conjunction with the term“comprising” in the claims and/or the specification may mean “one”, butit is also consistent with the meaning of “one or more”, “at least one”,and “one or more than one”. Similarly, the word “another” may mean atleast a second or more.

As used in this specification and claim(s), the words “comprising” (andany form of comprising, such as “comprise” and “comprises”), “having”(and any form of having, such as “have” and “has”), “including” (and anyform of including, such as “include” and “includes”) or “containing”(and any form of containing, such as “contain” and “contains”), areinclusive or open-ended and do not exclude additional, unrecitedelements.

A vision-assisted robotized depalletizer 10 according to an illustrativeembodiment will now be described with reference to FIG. 1.

The vision-assisted robotized depalletizer 10 comprises a palletunloading station 12 for receiving a pallet 14 of products 16, a visionsystem 18 for taking an image (not shown) of the top portion 20 of thepallet 14, a robot arm 22 equipped with a depalletizing tool 24 forpicking the products 16 one by one and for positioning the products 16on an output conveyor 26.

The expression “top portion of the pallet” should be construed in thedescription and in the claims as a portion of the pallet that includestop surfaces of products located in the pallet that are visible fromabove the pallet.

As an input, the depalletizer 10 receives a pallet 14 of products 16.

The robot arm 22 guided by the vision system 18 depalletizes theproducts 16 one by one and transfers them on the output conveyor 26 in apredetermined manner, yielding a flow of unitized products 16 aligned ina desired orientation on the output conveyor 26.

The expression “product” should be construed in the description and inthe claims as including any type of consumer goods in any type ofpackaging, such as, without limitations, closed cartons, totes, open topcartons, trays with or without shrink wrapped film, bags and pouches,etc.

The product dimensions may vary greatly between each different type ofproduct. Typical dimensions (W×L×H) are between 10.2 cm×15.2 cm×5.1 cm(4″×6″×2″) and 63.5 cm×63.5 cm×132 cm (25″×25″×52″).

Also, typically, there are no substantial gaps between the products 16when placed on the incoming pallet 14. It is to be noted however, that apallet that includes gaps between products can also be provided as aninput to the depalletizer 10 and processed thereby.

The pallet 14 loaded with products 16 overall dimensions also varysignificantly. The typical size of a loaded pallet 14 is (W×L×H) is 102cm×122 cm×183 cm (40″×48″×72″) but can vary from 82 cm×102 cm×122 cm(32″×40″×48″) to 112 cm×132 cm×244 cm (44″×52″×96″).

The pallet 14 to be depalletized is received at the pallet unloadingstation 12. According to the illustrated embodiment, the palletunloading station 12 is the infeed end 28 of a pallet conveyor 30.

The loaded pallet 14 is first deposited on a feeding conveyor 32 by anexternal system, such as a fork lift (not shown) or another conveyor.When the pallet unloading station 12 is empty, the feeding conveyor 32moves another loaded pallet 14 to the pallet unloading station 12 (seearrow 34).

After all products 16 have been depalletized, the empty pallet 33 ismoved by the conveyor 30 at its outfeed end 36. The empty pallet 33 isremoved using a fork lift (not shown), a conveyor (not shown), or anyother means to transport a pallet.

According to another embodiment (not shown), the unloading palletstation includes a pallet elevator that lifts the pallet as it isgetting depalettized.

According to still another embodiment (not shown), one or both of theconveyors 30 and 32 is/are omitted and the pallet unloading station 12is in the form of a table that receives the loaded pallet 14 directlyfrom a fork lift. According to such an embodiment, the empty pallet 33is removed from the table similarly.

Since conveyors and fork lift are believed to be well known in the art,they will not be described herein in more detail for concision purposes.

The vision system 18 includes one or more image acquisition sensors 19that are so positioned relative to the pallet unloading station 12 so asto have a field of view 38 covering the top portion 20 of the loadedpallet 14 at the pallet unloading station 12.

The sensors 19 are not limited to be positioned above the loaded pallet14 and, alternatively or additionally, one or more additional imageacquisition sensors (not shown) can be positioned for example diagonallyof the pallet unloading station 12 yielding a field of view including atleast part of one side of the loaded pallet 14.

The sensor 19 allows acquiring sufficient data to reconstruct an imageof the top 20 of the loaded pallet 14 and/or a three-dimension 3D modelthereof. Conventional industrial gray tone or color cameras and/or lasertriangulation scanners, structured light, stereoscopy or “time-of-flightcamera” such as Microsoft Kinect® sensors can be used.

Since such conventional cameras and time-of-flight type sensors arebelieved to be well known in the art, they will not be described hereinin more detail for concision purposes.

The vision system 18 further includes a controller 40 that is wired tothe sensors 19 or wirelessly coupled thereto and both are configured forthe transfer of acquired image data between the sensors 19 and thecontroller 40.

The controller 40 processes the received image data of the top portionof the pallet 14 and at least one characteristic of the products, suchas their theoretical or measured dimensions (length, width and height),expected container types, and/or any other characteristic that mayimpact the vision processing to determine the position of each product16 in the top portion of the pallet 14.

The product characteristics used by the controller 40 are predeterminedand inputted therein by a user or received thereby from a remotecomputer or controller (both not shown). More specifically, productdescriptions, characteristics and information thereabout can be storedin a database to be transferred through network communication to thecontroller 40 at time of entrance of the pallet 14.

The controller 40 is configured or programmed for combining andanalyzing the information acquired by the image acquisition sensors 19and the predetermined characteristics of the products 16 to detect theproducts' position relative to the robot 22. The following are nonlimitative examples of characteristics that are analyzed from the imageby the controller 40: edges of the products 16, corners of the products16, and pattern repetition of the products 16. These characteristics areobtained on the image for example by depth or intensity variationsthereon. The controller 40 then determines and sends to the robotcontroller (not shown) data indicative of the position and size of thenext product 16 to pick.

The position of the product to pick is referenced to the same point oforigin in the working space of both the controller 40 and the robot 22.

In some embodiment, the controller 40 and the robot controller are thesame.

The expression “controller” should be construed broadly as including oneor more electronic devices, including for example one or more computersthat are configured with components and/or programmed with instructionsthat produce one or more functionalities.

The robot 22 is in the form of a standard four (4) or six (6) axisindustrial articulated robot arm and is equipped with a standard orcustom designed end of arm tool 24 capable to securely pick and transferthe products from the loaded pallet to the output conveyor is used. Aconventional robot arm can be used, such as ABB®'s IRB 660 or IRB 6640,FANUC®'s R2000 or M410, or any similar robot arm offered by othermanufacturers such as Kuka™ or Motoman™.

According to another embodiment (not shown), one or more additionalrobot arm are used (not shown), for example when there are two outputconveyors or more.

The end of arm tool 24 is configured to pick from the loaded pallet 14 avariety of products and product dimensions. According to the illustratedembodiment, the end of arm tool 24 includes suction cups or suction pads42 that are used on the top and/or side of the product 16 to lift it anda bottom support plate or fingers 44 that can extend underneath theproduct 16 that is then secured to the tool 24 by the suction cups orpads 42.

According to another embodiment (not shown), the end of arm tool 24 usesonly suction cups or suction pads on the upper or side surface of theproduct 16 to lift and transfer it on the output conveyor 26. Accordingto another embodiment (not shown), the end of arm tool 24 uses twovertical side plates to clamp the products 14.

The output conveyor 26 is in the form of a linear conveyor where theproducts 16 are placed one by one by the depalletizer system 10.According to another embodiment (not shown), the output conveyor isreplaced by an output table or any other means adapted to receive theproducts, such as without limitations, an automated guided vehicle(AGV). According to still another embodiment (not shown), two outputconveyors (or more) or other output means are used.

The flow chart 50 shown at FIG. 4 describes an illustrated embodiment ofa depalletizing method.

With reference to FIGS. 1 to 4, the operation of the depalletizingsystem 10 according to the first embodiment will now be described inmore detail.

FIG. 1 shows a loaded pallet 14 being forwarded next to the robot 22 inthe loaded pallet station 12 (see arrow 34), ready to be depalletized(step 52). The nominal information about the products 16 on the pallet14 is inputted to the controller 40 (step 54).

The vision system 18 takes an image of the top portion of the pallet 16(step 56) and the controller 40 determines the first product 16′ to beremoved (step 58) and its position. The controller 40 then sends to therobot controller the data indicative of the position and size of theproduct 16′ to pick (step 60).

As can be seen in FIG. 2, this first product 16′ is transferred on theoutput conveyor 26 by the robot arm 22 using the end of arm tool 24(step 62).

The vision system 18 takes a new picture of the top portion of thepallet 14 (step 56) located in the loaded pallet station 12, thecontroller 40 verifies whether the pallet 14 is empty (step 64) and ifthe pallet 14 is not empty then the controller 40 identifies the nextproduct 16 to be removed (step 58).

If the pallet 14 is determined to be empty by the controller 40, anotherpallet 14 is moved at the pallet unloading station 12 while the emptypallet is removed by the conveyor 30 and the method start anew.

Turning now to FIG. 3, the second and third products 16″ and 16″′ areshown transferred by the robot arm 22 and the end of arm tool 24 on theoutput conveyor 26 and moving towards another station or system (notshown) (arrow 46).

The output conveyor 26 remains in operation while the pallet 14 isdepalletized and while a loaded pallet 14 is brought in the palletunloading station 12 and an empty pallet 30 is removed therefrom by thepallet conveyor 30.

Anyone of the illustrated conveyors 26, 30 and 32 can be of the rollertype or of the belt type and can be motorized or not.

While it is not illustrated in FIGS. 1 to 3, a slip sheet removalequipment (not shown) can be used to remove any pallet layer separating(also known as “tier sheet”, “separator sheet” and “slip sheet”) sheet48 present in the pallet 14. Such an equipment can be seen as being partof the depalletizer system 10 or not. In some embodiment, the tool 24can be equipped, for example, of a side suction pad that can be used toremove the slip sheet 48. The vision system determines the presence orabsence of a slip sheet.

Since slip sheet removal equipment are believed to be well known in theart, it will not be described herein in more detail for concisionpurposes.

According to another embodiment, no information about the products 16 topick is used to the vision system and the vision system 18 determinesthe position of a product to pick based on the one or more images ittakes. Step 54 is then omitted in the method 50.

According to still another embodiment (not shown), one of a plurality ofdifferent tools, each adapted for a different type of product, is usedby the robot to depalletize the products. According to such anembodiment, each tool can be provided with a tool changer so that therobot can automatically change the tool when needed. The moreappropriate end of arm tool 24 to use is based for example on the sizeand type of the next product to pick.

It is to be noted that many other modifications could be made to therobotized depalletizer described hereinabove and illustrated in theappended drawings. For example:

-   -   the end of arm tooling could include only suction cups, only        mechanical means or a combination of both to pick and transfer        the products;    -   two output conveyors or more could be installed, for example        when there is one or two robot arms in the system; and    -   the articulated robot arm could be replaced by a gantry type        system.

A person skilled in the art will now appreciate that an illustratedembodiment of a depalletizer can be used to depalletize a full orpartially full pallet, that includes a same or different types ofproducts.

It is to be understood that illustrative embodiments of the depalletizerare not limited in their application to the details of construction andparts illustrated in the accompanying drawings and describedhereinabove. Other embodiments can be foreseen and practiced in variousways. It is also to be understood that the phraseology or terminologyused herein is for the purpose of description and not limitation.

What is claimed is:
 1. A depalletizer comprising: a pallet unloadingstation for receiving a pallet of products; a vision system for takingat least one image of a top portion of the pallet received at the palletstation, and for using said at least one image for determining theposition of said each product in the top portion of the pallet; and arobot equipped with a depalletizing tool; the robot being coupled to thevision system for receiving information therefrom indicative of theposition of said each product from the top portion and for using thatinformation to pick and position on an output surface said each productfrom the top portion.
 2. The depalletizer of claim 1, wherein the visionsystem includes at least one image acquisition sensor.
 3. Thedepalletizer of claim 2, wherein the vision system includes a visionsystem controller coupled i) to the at least one image acquisitionsensor for receiving the at least one image therefrom and ii) to thevision system controller for sending thereto the information indicativeof the position of said each product from the top portion.
 4. Thedepalletizer of claim 3, wherein said determining the position of saideach product is achieved by the controller analysing at least one of thefollowing characteristics from the at least one image: edges of productsseen on the at least one image, corners of the products seen on the atleast one image, and pattern repetition of the products seen on the atleast one image.
 5. The depalletizer of claim 3, wherein saiddetermining the position of said each product is achieved by thecontroller analysing at least one of depth and intensity variations ofthe at least one image.
 6. The depalletizer of claim 3, wherein thevision system controller further uses at least one characteristic of theproducts along with said at least one image in determining the positionof said each product from the top portion of the pallet.
 7. Thedepalletizer of claim 6, wherein the at least one characteristic of theproducts includes at least one of i) a dimension of the products and ii)a type of products.
 8. The depalletizer of claim 2, wherein the at leastone image acquisition sensor includes at least one of a gray tonecamera, a colour camera, a laser triangulation scanner, a structuredlight type sensor, a stereoscopy sensor, and time-of-flight type sensor.9. The depalletizer of claim 2, wherein the at least one imageacquisition sensor is positioned above the pallet unloading station. 10.The depalletizer of claim 2, wherein the at least one image acquisitionsensor includes a plurality of sensors.
 11. The depalletizer of claim 1,wherein the output surface is a surface of at least one conveyor. 12.The depalletizer of claim 1, wherein the pallet unloading station is asurface of a conveyor.
 13. The depalletizer of claim 1, wherein theproducts include one of close cartons, totes, open top cartons, traysbags and pouches.
 14. The depalletizer of claim 1, wherein the robot isa four (4) or six (6) axis articulated robot arm.
 15. The depalletizerof claim 1, wherein the robot is a gantry-type robot system.
 16. Thedepalletizer of claim 1, wherein the depalletizing tool includes atleast one of a suction cup, a suction pad, a support plate and supportfingers.
 17. The depalletizer of claim 1, wherein the depalletizing toolis part of a series of depalletizing tools, each mountable via a toolchanger and being adapted to a specific type of products.
 18. Thedepalletizer of claim 1, wherein the vision system further uses the atleast one image for detecting a slip sheet in the pallet; thedepalletizer further comprising a slip-sheet removing device coupled tothe vision system for removing the slip sheet.
 19. The depalletizer ofclaim 18, wherein the slip-sheet removing device is part of thedepalletizing tool.
 20. The depalletizer of claim 1, wherein the palletunloading station includes of a pallet elevator that lifts the pallet ofproducts as it is depalettized.
 21. A depalletizer comprising: a visionsystem including at least one image acquisition sensor for taking animage of a top portion of a pallet positioned under the at least oneimage acquisition system, and a vision system controller coupled to theat least one image acquisition sensor for receiving the image and forusing said image to determine the position of each product in the topportion of the pallet; and a robot equipped with a depalletizing toolincluding at least one suction means; the robot being coupled to thevision system controller for receiving information therefrom indicativeof the position of each product from the top portion and for using thatinformation to pick and position on an output conveyor said each productfrom the top portion.
 22. A method for depalletizing a pallet ofproducts, the method comprising: taking an image of a top portion of thepallet; the controller using the image to determine a position of aproduct to pick from the top portion of the pallet; the controllersending to a robot information indicative of the position of the productto pick on the pallet; and the robot picking the product and moving theproduct on an output surface.
 23. A method as recited in claim 22,further comprising: inputting in a controller information about theproducts; and the controller further using the information about theproducts along with the image to determine the position of the productto pick from the top portion of the pallet.